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Abstract. Finding feasible points for which the proof succeeds is a crit- 
ical issue in safe Branch and Bound algorithms which handle continuous 
problems. In this paper, we introduce a new strategy to compute very 
accurate approximations of feasible points. This strategy takes advan- 
tage of the Newton method for under-constrained systems of equations 
and inequalities. More precisely, it exploits the optimal solution of a lin- 
ear relaxation of the problem to compute efficiently a promising upper 
bound. First experiments on the Coconuts benchmarks demonstrate that 
this approach is very effective. 
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CO ■ Optimization problems are a challenge for CP in finite domains; they are also 

a big challenge for CP on continuous domains. The point is that CP solvers 

f*"*- ■ are much slower than classical (non-safe) mathematical methods on nonlinear 

constraint problems as soon as we consider optimization problems. The tech- 
niques introduced in this paper try to boost constraints techniques on these 
problems and thus, to reduce the gap between efficient but unsafe systems like 
BARON 4 , and the slow but safe constraint based approaches. We consider here 

k>( \ the global optimization problem V to minimize an objective function under non- 

$_j ■ linear equalities and inequalities, 



minimize f{x) 

subject to gi{x) = 0, i € {1, .., k} (1) 

hj{x)<0, j £{!,.. ,m} 

with x £ x, / : M n -* M, g. t : FT -> M and hj : M n -> M; Functions 
/ , gi and hj are nonlinear and continuously differentiable on some vector x of 
intervals of 1R. For convenience, in the sequel, g{x) (resp. h{x)) will denote the 
vector of gi(x) (resp. hj(x)) functions. 



* An extented version of this paper is available at: 

http://www.i3s.unice.fr/%7Emh/RR/2008/RR-08.ll-A.GOLDSZTEJN.pdf 
4 See http://www.andrew.cmu.edu/user/nslb/baron/baron.html 



Algorithm 1 Branch and Bound algorithm 



Function BB(IN x, e; OUT S, [L,U]) 

% S: set of proven feasible points 

% f x denotes the set of possible values for / in x 

% nbStarts: number of starting points in the first upper-bounding 

£^{x}; (L, [/)<—(— 00, +oo); S^UpperBounding(x' , nbStarts); 

while w([L,U]) >edo 

x'<— x" such that f x ,/ = mm{f x „ : x" 6 £}; £^£\x'; f x /<— mm(f x >, U); 

x'^Prune(x'); f x /<— Lower -Bound(x'); <S<— 5 U Upper Bounding (x' ,1); 

ifx'^0 then (x'^x^SpZi^x'); £^£ U {xi, x' 2 }; 

if£ = then (L,[/)<-(+oo, -oo) 

else (L, J7)+-(mm{f x „ : x" e £},mm{f x ,/ : x" G 5}) 
endwhile 



The difficulties in such global optimization problems come mainly from the 
fact that many local minimizers may exist but only few of them are global min- 
imizers [3]. Moreover, the feasible region may be disconnected. Thus, finding 
feasible points is a critical issue in safe Branch and Bound algorithms for con- 
tinuous global optimization. Standard strategies use local search techniques to 
provide a reasonable approximation of an upper bound and try to prove that a 
feasible solution actually exists within the box around the guessed global opti- 
mum. Practically, finding a guessed point for which the proof succeeds is often 
a very costly process. 

In this paper, we introduce a new strategy to compute very accurate approx- 
imations of feasible points. This strategy takes advantage of the Newton method 
for under-constrained systems of equations and inequalities. More precisely, this 
procedure exploits the optimal solution of a linear relaxation of the problem 
to compute efficiently a promising upper bound. First experiments on the Co- 
conuts benchmarks demonstrate that the combination of this procedure with a 
safe Branch and Bound algorithm drastically improves the performances. 

The Branch and Bound schema 

The algorithm (see Algorithm 1) we describe here is derived from the well known 
Branch and Bound schema introduced by Horst and Tuy for finding a global min- 
imizer. Interval analysis techniques are used to ensure rigorous and safe com- 
putations whereas constraint programming techniques are used to improve the 
reduction of the feasible space. 

Algorithm 1 computes enclosers for minimizers and safe bounds of the global 
minimum value within an initial box x. Algorithm 1 maintains two lists : a list 
C of boxes to be processed and a list S of proven feasible boxes. It provides a 
rigorous encloser [L, U] of the global optimum with respect to a tolerance e. 

Algorithm 1 starts with UpperBounding(x, nbStarts) which computes a set 
of feasible boxes by calling a local search with nbStarts starting points and a 
proof procedure. 



The box around the local solution is added to S if it is proved to contain 
a feasible point. At this stage, if the box x' is empty then, either it does not 
contain any feasible point or its lower bound f x / is greater than the current 
upper bound U. If x' is not empty, the box is split along one of the variables 5 
of the problem. 

In the main loop, algorithm 1 selects the box with the lowest lower bound 
of the objective function. The Prune function applies filtering techniques to 
reduce the size of the box x'. In the framework we have implemented, Prune just 
uses a 2B-filtering algorithm [2]. Then, Lower Bound(x.') computes a rigorous 
lower bound f x / using a linear programming relaxation of the initial problem. 
Actually, function LowerBound is based on the linearization techniques of the 
Quad- framework [1]. LowerBound computes a safe minimizer f x / thanks to the 
techniques introduced by Neumaier et al. 

Algorithm 1 maintains the lowest lower bound L of the remaining boxes £ 
and the lowest upper bound U of proven feasible boxes. The algorithm terminates 
when the space between U and L becomes smaller than the given tolerance e. 

The Upper-bounding step (see Algorithm 2) performs a multistart strategy 
where a set of nbStarts starting points are provided to a local optimization 
solver. The solutions computed by the local solver are then given to a function 
InflateAndProve which uses an existence proof procedure based on the Borsuk 
test. However, the proof procedure may fail to prove the existence of a feasible 
point within box x p . The most common source of failure is that the "guess" 
provided by the local search lies too far from the feasible region. 

A new upper bounding strategy 

The upper bounding procedure described in the previous section relies on a local 
search to provide a "guessed" feasible point lying in the neighborhood of a local 
optima. However, the effects of floating point computation on the provided local 
optima are hard to predict. As a result, the local optima might lie outside the 
feasible region and the proof procedure might fail to build a proven box around 
this point. 

We propose here a new upper bounding strategy which attempts to take ad- 
vantage of the solution of a linear outer approximation of the problem. The lower 
bound process uses such an approximation to compute a safe lower bound of V . 
When the LP is solved, a solution xlp is always computed and, thus, available 
for free. This solution being an optimal solution of an outer approximation of V, 
it lies outside the feasible region. Thus, xlp is not a feasible point. Nevertheless, 
xlp may be a good starting point to consider for the following reasons: 

- At each iteration, the branch and bound process splits the domain of the 
variables. The smaller the box is, the nearest xlp is from the actual optima 
of V. 

— The proof process inflates a box around the initial guess. This process may 
compensate the effect of the distance of xlp from the feasible region. 



Various heuristics are used to select the variable the domain of which has to be split. 



Algorithm 2 Upper bounding build from the LP optimal solution x* LP 
Function UpperBounding(lW x, x* LP , nbStarts; OUT S') 

% S'\ list of proven feasible boxes; nbStarts: number of starting points 

% x* LP : the optimal solution of the LP relaxation of 'P(x) 

S' <— 0; x* corr ^FeasibilityCorrection(a;| y p); x p ^InnateAndProve(:r* orr , x ); 

if x p / then S' <-S' U x p 

return S 



However, while xlp converges to a feasible point, the process might be quite 
slow. To speed up the upper bounding process, we have introduced a light 
weight, though efficient, procedure which compute a feasible point from a point 
lying in the neighborhood of the feasible region. This procedure which is called 
F easibilityC orrection will be detailed in the next subsection. 

Algorithm 2 describes how an upper bound may be build from the solution 
of the linear problem used in the lower bounding procedure. 



Computing pseudo-feasible points 



This section introduces an adaptation of the Newton method to under-constrained 
systems of equations and inequalities which provides very accurate approxima- 
tions of feasible points at a low computational cost. When the system of equa- 
tions g(x) = is under-constrained there is a manifold of solutions. l(x), the 
linear approximation is still valid in this situation, but the linear system of 
equations l(x) = is now under-constrained, and has therefore an afHne space of 
solutions. So we have to choose a solution x^ of the linearized equation l(x) = 
among the afHne space of solutions. As x^> is supposed to be an approximate 
solution of g(x) = 0, the best choice is certainly the solution of l(x) = which is 
the closest to x^ ' . This solution can easily be computed with the Moore-Penrose 
inverse: x^ = x*- ' — A^(x^°')g(x^ '), where A+ £ M nxm is the Moore-Penrose 
inverse of A g G lR mxn , the solution of the linearized equation which minimizes 
II^U) _2;(o)||_ Applying previous relation recursively leads to a sequence of vec- 
tors which converges to a solution close to the initial approximation, provided 
that this latter is accurate enough. 

The Moore-Penrose inverse can be computed in several ways: a singular value 
decomposition can be used, or in the case where A g has full row rank (which is 
the case for A g (x^') if a^ -* is non-singular) the Moore-Penrose inverse can be 
computed using A+ = A^(AgA^ ) _1 . 

Inequality constraints are changed to equalities by introducing slack vari- 
ables: hj(x) < <=? hj{x) = —sf. So the Newton method for under- 
constrained systems of equations can be applied. 



Experiments 

In this Section, we comment the results of the experiments with our new upper 
bounding strategy on a significant set of benchmarks. Detailled results can be 
found in the resarch report ISRN I3S/RR-2008-11-FR 6 ). All the benchmarks 
come from the collection of benchmarks of the Coconuts project 7 . We have se- 
lected 35 benchmarks where Icos succeeds to find the global minimum while 
relying on an unsafe local search. We did compare our new upper bounding 
strategy with the following upper bounding strategies: 

SI: This strategy directly uses the guess from the local search, i.e. this strategy 
uses a simplified version of algorithm 1 where the proof procedure has been 
dropped. As a consequence, it does not suffer from the difficulty to prove 
the existence of a feasible point. However, this strategy is unsafe and may 
produce wrong results. 

S2: This strategy attempts to prove the existence of a feasible point within a 
box build around the local search guess. Here, all provided solutions are safe 
and the solving process is rigorous. 

S3: Our upper bounding strategy where the upper bounding relies on the optimal 
solution of the problem linear relaxation to build a box proved to hold a 
feasible point. A call to the correction procedure attempts to compensate 
the effect of the outer approximation. 

S4: This strategy applies the correction procedure to the output of the local 
search (to improve the approximate solution given by a local search). 

S5: This strategy mainly differs from S3 by the fact that it does not call the 
correction procedure 

S3, our new upper bounding strategy is the best strategy: 31 benchmarks are 
now solved within the 30s time out; moreover, almost all benchmarks are solved 
in much less time and with a greater amount of proven solutions. This new 
strategy improves drastically the performance of the upper bounding procedure 
and competes well with a local search. 

Current work aims at improving and generalizing this framework and its 
implementation. 
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